Video encoders are now commonly used in consumer end products like DVD reader/writers at consumer affordable prices. Video encoders have also been increasingly found in server software applications for real time internet video streaming. However, these video encoders often do not perform up to par as compared with professional encoders. The resulting poorly encoded video images often contain video artifacts such as blocking, corner outlier, and ringing.
Besides the effects of encoding technology of the video encoders, video pre-processing and post-processing are also performed to improve the quality of the encoded video. Video post-processing processes the decoded video from the bitstream while video pre-processing processes input video data prior to video encoding.
Due to competitiveness in the market of video encoders, low cost solutions with inter-operability of video decoders are preferred. As such, use of video pre-processing is an important method for reducing encoding complexity and for improving video quality for video encoders. This is especially so when the bitstream is decoded by third party video players. It is also desired that the pre-processing solution can be built into existing encoder solutions or with minimal hardware modification such that the system cost is not significantly increased.
Some existing techniques for pre-processing apply pre-filtering on the residual image rather than the image itself. This may result in error residual propagation in a group of pictures giving undesired defects in especially lengthy groups. In addition, spatial filtering is difficult to apply within the video encoder as it limits the filtering in pre-defined block sizes.
Segall, C. A. and Katsaggelos, A. K., “Pre- and Post-processing Algorithms for Compressed Video Enhancement”, provides a good summary of the existing techniques related to pre-processing video data for compressed video enhancement. The different techniques include operating directly on intensity data of original image using for example a low pass filter, operating inside the encoder and using quantized DCT coefficients to remove insignificant features. The disclosed work involves pre-filtering the residual between predicted and current frame with response of filter coupled to quantization scale and distortion metric on blocking artifacts minimized.
In U.S. Pat. No. 6,037,986, “Video Pre-processing Method and Apparatus with Selective Filtering based on Motion Detection”, by Zhang et al. describes an adaptive low pass vertical filter (7 taps) selected from predetermined filter coefficients using pre-compression data like bit-rate range, motion metric, frame wise motion and bit-rate or post-compression data like moving average of quantization, sum of absolute motion vectors, number of bits per compressed frame.
In U.S. Pat. No. 6,281,942, “Spatial and Temporal Filtering Mechanism for Digital Motion Video Signals”, by Wang, a technique operating a spatial filter using a 4×4 block with control of heavy, light or no filter dependency on thresholding of block variance is disclosed.
In U.S. Pat. No. 5,231,484, “Motion Video Compression System with Adaptive Bit Allocation and Quantization”, Gonzales et al. describes a pre-processing method that selects between 2 sets of 3-tap filters or none according to the degree of quantization.
Other existing methods may also use motion estimation engines to perform motion compensated 3D spatial and temporal filtering. When motion estimation is incorrect, these motion compensated techniques disadvantageously create undesired artifacts such as smearing regions. However, additional motion estimators for correcting these artifacts will disadvantageously increase the costs of these pre-processing solutions.
Pre-filtering may also result in the loss of video resolution, giving reduction in sharpness of image as filter strength of the pre-filters increases. Often the blurring of the image is not compensated by the gain in encoding efficiency.
Furthermore, variable delayed response between filtering and feedback of post-compression parameters are often not considered. In situations of long delay especially when parameters, which are affected by pre-filtering, are used, oscillations between extreme filtering states may occur, leading to instability in video encoding.
Therefore, a need exists for an improved method for improving visual quality of video especially in difficult video sequences with fast motion and low bit rate applications